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METHOD AND INSTALLATION FOR DETECTING AND FOLLOWING AN EYE 

AND THE GAZE DIRECTION THEREOF 
RELATED APPLICATION 

This application claims priority and benefit ficom Swedish patent application No. 0203457-7, 
5 filed November 21, 2002, tiie entire teachings of vMch are incorporated herein by reference. 
TECHNICAL FIELD 

The present invention relates a method and an installation for detecting and tracking eyes 
and gaze angles/directions. Such a system can for example be used for observing or determining 
the position on a monitor or display at which a computer user is looking. 

10 BACKGROUND 

Monitoring or tracking eye movements and gaze points can be used in many different coii- 
texts. Information about the position at which a person is looking can be used for analyzing the 
behavior and consciousness of the person. It can be used both for evaluating the object at which 
the person is looking and for evaluating the respective person. The fields of use are many and 

15 thereamong studies on the usability of software and different types of interfaces, evaluations of 
home pages, advertising and advertisements, means for educating pilots in simulator surroimd- 
ings, research in psychology, behavior science and human perception and diagnosis of various 
^es of visual faults and illnesses can be mentioned. 

In addition to these applications also interactive applications exist \\^ch employ informa- 

20 tion about tiie place at which a person is looking in order to respond or react in different ways. A 
physically disabled person can for example interact with a computer by the procedure that those 
objects on the monitor of the computer at which he is looking are activated. In an arcade game 
tile adventure experience of the game can be very much enhanced by the procedure that those 
objects at -wldch. a person is looking are made to be centered in the image or by the procedure tibat 

25 the person is allowed to direct a weapon using her/his eyes. In the same way the advertising or 
display for Christmas in a shop window can react to fixe feet that a person is looking at it Inter- 
faces for for example a computer could utilize continuous information about the position at which 
a user is looking to be better capable of displaying the object in which a us« is interested and of 
adaptmg m an mteUigent way to different behaviors of the user. A motorcar tiiat receives infor- 

30 mation about the position at which the driver is lookmg could issue an alarm in the case where 
the driver is tired, distracted or intoxicated. Military appUcations for directing weapons or for 
steering vehicles using eye movements have also been developed. These are only a small number 
of the fields of use that can be of mterest for an installation that can detect and track eyes and 
gaze angles. 

35 Many different technologies exist that can be used for tracking an eye and the gaze angle 
thereof A first technology is to use a photosensor and a Ught source, the light of which is re- 
flected fi:om the cornea and from the lens of an eye and thereby causes four different reflections. 
The reflection tiiereof having the strongest light intensity is tiie outer cornea reflection or glint, 
also called tiie first Purkinje reflection. Thereafter the second, tiie third and fourth Puridnje re- 
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flections follow which correspond to reflections in the inner surface of the cornea, the outer sur- 
face of tlxe lens and the inner surfece of the lens. The first and fourth Purkinje-reflections are lo- 
cated in the same focal plane and the relationship of the positions of these two reflections varies 
with rotation of the eye. 

5 A second technology for tracking an eye and the gaze angle thereof is to detect, using some 
type of photosensor, the boundary line between the white or sclera and the iris of an eye. A third 
technology is to detect the face of person using a camera and to indirectly deduce, by observing 
how fbs eyes move in relation to the face, the direction in which the person is looking. A fourth 
technology is to use that the cornea and the retina have different electrical potentials for measur- 
10 ing rotation of an eye. A fifth technology is to use a contact lens fitted with suitable devices to be 
capable of measuring how the contact lens and hence also the eye rotates. A sixth technology is to 
measure, using a photosensor, the ellipticity of the pupil or of the iris when the eye rotates. A 
seventh technology is to use a diode and two or more optical transistors mounted in a pair of 
spectacles. When the light firom the diode is reflected JBrom the cornea and then hits an optical 
15 transistor the signals provided by the transistor will vary in accordance witii ttie travel distance of 
the light from tiie diode via the cornea and to the transistor. An eight technology is to use a pho- 
tosensor to detect the reflection of a light source from tide cornea and then to observe how the po- 
sition thereof is related to the center of the pupil or iris. The pupil can in this case be made more 
easily observable by using illumination devices mounted close to Hos photosensor illuminating 
20 the retina of the eye and thereby causmg a bright eye effect or by illuminating the eye so that the 
iris is made bright whereas the piq>il remains dark. 

In U.S. patent document 5,861,940 a system for detecting eyes and for tracking gaze angles 
is disclosed. The system includes light sources comprismg an IR-laser 2 that is provided with a 
suitable optical system and issues a convergent light beam with a focus in front of the eye and 
25 two IR-lasers or IR-diodes 9, 10 that altematingly emit divergent light beams. By detecting the 
reflection from the retina using a position sensitive detector information about the position of the 
eye is obtained. By detecting the reflections of a plurality of light sources from the cornea the 
distance of the eye from the video camera is determined. The latter two IR-lasers are placed at the 
bottom edge of a computer monitor 1. Other systems are disclosed in U.S. patent documents 
30 6,079,829, 6,152,563 and 6,246,779. 

In automatic systems that employ a video camora it can be difficult to determme the posi- 
tions in a captured picture, where reflections of light sources from the eyes of a user are located. 
This depends among other things on the feet the user can move her/his head, displace it laterally 
or to places at a smaller or larger distance of flie camera. The automatic system can contain com- 
as plex image processing and inu^e recognition routines for determining those areas where images 
of eyes are located. Hence there is a need for methods for facilitating such determinations of 
areas in a captured picture in which reflections and images of eyes are located. Furthermore, in 
determimng the pomt of regard on a surface such as a monitor, always the distance between the 
eyes of the user and flie monitor must be determined and thus efficient methods for such a deter- 
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mination are required. 

SUMMARY OF THE INVENTION 

It is an object of the invention to provide an efficient method and an efficient instaUation 
for detectiag and tracking eyes and gaze angles/directions. 
5 In a method and in a device for detecting eyes and gaze angles/directions a photosensor 
such as a CCD-unit or CMOS-unit provided with a suitable optical system or a video camera, one 
or more Ught sources and a calculation and control unit are used. To determine the position on for 
example a monitor at vMch a person is looking the position of the eyes of the person, both in a 
depth direction and in lateral directions, as seen from the monitor, and the direction in which the 
10 eye is looking, must be known. If only the angle and the position in two dimensions are known, a 
parallax distortion will evolve when the eye moves in the depth direction. This parallax distortion 
is often significantly larger than what is acceptable in many applications. 

To determine the position of an eye, i.e. the position thereof m relation to a photosensor or 
a monitor, the eye is illuminated using two or more light sources arranged at a distance of each 
15 other. The images of the reflections of these Ught sources from the person's cornea on to the 
photosensor will then depend on lixe place in the field of vision of the photosensor at which Ihe 
person is located. The images of the reflections on the photosensor provide all the information re- 
quired to exactiy determine the place where the person is located in the field of vision of the 
photosensor. If it is assumed that the cornea of the eye is spherical this function is unambiguously 
20 defined and is invertible already in the case where the number of light sources is as small as two, 
but also using more advanced assumptions a good accuracy and reliability in the determmation of 
the place where the eyes are located are obtained by evaluating the whole pattern that is formed 
j&om reflections of a pluraUty of Ught sources, i.e. the positions of the reflections in relation to 
each other and in the whole picture. In a simple case only the size of the pattern and the position 
25 thereof in the captured picture are determined. 

Thus, in Ihe method a photosensor is used for creating a two-dimensional picture of the 
reaUty and'generally one or more Ught sources. The light source/light sources Uluminate possible 
eyes within the visual field of the photosensor so that diffusely reflecting surfaces become more 
easUy observable to the photosensor and so that Ught firom the Ught source/Ught sources reaching 
30 corneas causes reflections which by the photosensor are experienced as discrete bright pointe. 

The two-dimensional picture created by the photosensor is analyzed using complex image 
processing to determine the position in the picture where possible eyes and reflections of light are 
located. This information is then used to determine where eyes are located and/or in which direc- 
tions Ihey are looking, which in turn can be used for for example determining the position on a 
35 monitor at which a computer user is looking. 

In Ihe two-dimensional picture pupils and/or irises are searched for. The image of the pupil 
can be made more easily observable by Uluminating it by one or more Ught sources. The Ught 
source/Ught sources can be placed at a distance fi»m the optical system that focuses the two-di- 
mensional picture on the photosensor. It results in that only a smaU portion of the iUuminating 
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light will enter the eye and hence that the pupil will appear as darker than the surrounding face. In 
that way the boundary line between the pupil and the iris can be more easily detected than in the 
case vdiere no illumination is used. 

An alternative to tiie illumination mentioned above comprises that the light source/light 
5 sources are placed very close to or on the optical system that focuses the two-dimensional picture 
on the photosensor. The diffused field of the light source/light sources then corresponds to the 
field of vision of the photosensor, and to an eye that is located within the field of vision of the 
photosensor tiie point on the retina that is illuminated by the light source/light sources will be the 
same or at least partly the same point that is imaged on the photosensor. It results in that the pupil 
10 appears to be brighter than the svirroundings thereof. The phenomenon is the same one that results 
in that eyes on photographs can appear to have a reddish color. 

To achieve the possibly strongest bright eye effect the light source/light sources are 
mounted as close as possible to the visual axis of the photosensor without interfering with the 
field of vision of the sensor. The portion of the retina of an eye that the photosensor is seeing then 
15 largely coincides with the portion of the retina that is illuminated by the light source/light 
sources. Provided that the light source/light sources must be placed in firont of the optical system 
that focuses the picture on the photosensor, a light source having a central hole of the same shape 
as the photosensor is a good way of arranging the light sourceAight sources as close to the visual 
axis of the photosensor without mterfering with the visual field thereof. The larger distance fi:om 
20 the optical system the light source is placed the larger the hole must be in order not to interfere 
with the visual field of the photosensor. Therefore it is desirable to be capable of placing the li^t 
soxirce as close to the optical system as possible in the case where a bright eye effect is aimed at 

Using the bright eye effect for determining the direction of eyes, at the same time as the 
positions of eyes are determined using a plurality of light sources that are reflected, can cause 
25 problems. The bright eye effect is achieved by using an illumination device/light source that is 
placed as close to the photosensor as possible. An illxmiination device placed at a distance from 
this light source will illuminate everything in addition to the pupil of the eye that is located within 
the visual field of the photosensor and hence reduce the contrast between the pupil and the iris in 
pictures captured by the photosensor. In addition there is a high probability that the reflection of 
30 this additional light soxirce in the picture will be located precisely at the boundary line between 
the pupil and iris which even more makes the search for this transition difficult. As seen fix)m the 
opposite perspective, that the position of the eye is to be determined, the bright eye effect that is 
used to bring out or enhance the pupil results in that the contrast of the transition region in the 
picture between the pupil and a reflection fi*om the cornea in firont of the pupil becomes lower 
35 compared to the case in which the pupil had not been illuminated using tiie bright eye effect. 
Thus, there is a risk that it could be difficult to observe reflections located in images of the illu- 
minated pupil. To avoid these problems two light settings, one for detemuning the direction and 
another for determining the position, can be alternated with each other so that only one is active 
for each captured picture. This implies that both the direction and flie position cannot be deter- 
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mined fix)m each picture but on the other hand it increases the robustness of the system. 

If the light source/light sources instead is/are placed at a distance from the optical axis of 
the photosensor a dark pupil is obtained whereas the surrounding face becomes brighter the 
stronger the light sources/light sources radiates/radiate. This light setting is preferred in the case 
5 where the iris to be found and it also has the advantage that the contrast between the pupil and the 
iris is not made lower or worse by diffuse light from the surroundings as is the case for a system 
based on the bright eye effect. 

In order not to disturb the person that is illuminated by the light source/light sources the 
light source/light sources can be used to emit light having wavelengths that are not visible to the 
10 human eye but that can be detected by the photosensor. For example, light in NIR-range, (NIR = 
Near InfraRed) can be used, i.e. light having wavelengths somewhat longer than what can be per- 
ceived by a human eye. Such light can be detected by most photosensors designed to detect visi- 
ble Ught. 

The photosensor used can advantageously be a high resolving type/high resolution type. It 

15 results in the fact that a large amount of data is obtained when the whole photosensor is exposed. 
In order to reduce the resources used in processing data from only a given area of interest (AOI) 
can be selected to be processed. Thus, to reduce the load on the communication link between the 
photosensor and tiie calculation unit the calculation unit can select to only ask the photosensor to 
provide those data that come from the current AOL In that way both calculation and communica- 

20 tion resources can be released for other purposes. To be capable of selecting a proper AOI for an 
exposing of the photosensor advantageously information about tiie positions where the eye has 
been located in the directly preceding pictures is used. 

The greatest advantage is obtained if as large portion of uimecessary picture information as 
possible is discarded as early as possible in the procedure. If the photosensor is made to only ex- 

25 pose or forward a portion of the information from the sensor svuface to the evaluation imit new 
possibilities evolve that previously were not possible. In that way photosensors having a very 
high resolution can be used without causing that the procedure operates slower. Instead of camera 
interfaces that are advanced, dijSicult to initiate and costly, standardized PC-buses can be used to 
obtain a sufficient transmission rate from the photosensor to the evaluation unit. Also the very 

30 exposing and the reading out of the photosensor data in many cases require a shorter time period 
if only a selected portion of the sensor is exposed. It results in that a larger velocity can be ob- 
tamed in the determination of the gaze point for tiie eye that is tracked. This is valuable in many 
applications such as for example in psychological studies. 

Additional objects and advantages of the invention will be set forth in the description which 

35 follows, and in part will be obvious from the description, or may be learned by practice of the in- 
vention. The objects and advantages of the invention may be realized and obtained by means of 
the methods, processes, instrumentalities and combinations particularly pointed out in the ap- 
pended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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While the novel features of the invention are set forth with particularly in the appended 
claims, a complete understanding of the invention, both as to orgam2ation and content, and of the 
above and other features thereof may be gained from and the invention will be better appreciated 
from a consideration of the foUowing detailed description of non-limitmg embodiments presented 
5 hereinbelow with reference to the accompanying drawings, in which: 

- Fig. lis a schematic picture from the front of an embodiment of an mstallation for detecting and 
tracking eyes and gaze ai^es, 

- Fig. 2 is a schematic diagram of the dependence, in a two-dimensional picture captured by a 
photosensor, of the distance between two Ught sources reflected from a cornea on the distance of 

10 the cornea from the photosensor, 

- Figs. 3a and 3b are schematic pictures from the front of an example of an embodiment of an 
iUumination device that is coaxial with the visual field of the photosensor for achieving a bright 
eye effect when different diodes are switched on and off, 

- Fig. 4a is a picture of an eye captured using a photosensor and illuminated in order to make it 
15 possible to determine the gaze angle of the eye and the place on which it is located in the two-di- 
mensional coordinate system of the photosensor, 

- Fig. 4b is a picture obtained using boundary line analysis of the picture of Fig. 4a, 

- Fig. 5a is a picture of an eye similar to Fig. 4a but illuminated for making it possible to deter- 
mine the distance of the eye from the photosensor, 

20 - Fig. 5b is a picture obtained using boundary line analysis of the picture of Fig. 5a, 

- Fig. 6 is a flow diagram illustratmg steps in processing mformation about found eyes in a two- 
dimensional picture for determining which eye is the right or the left one of a user, and 

- Fig. 7 is a flow diagram of the total procedure that is performed in an installation for determin- 
ing the gaze direction and the position of an eye. 

25 DETAILED DESCRIPTION 

In Fig. 1 an mstallation for determining the point on a monitor 1 at which a computer user 
is looking/gazing is shown. The determination is performed by tracking the eyes of the user and 
in particular by determining the gaze angles of the eyes and the distance of the eyes from the 
monitor. The installation includes the monitor 1, three identical light sources 2 mounted along a 

30 straight line at the upper edge of the monitor 1, a photosensor 4 placed at the center of the bottom 
edge of the monitor and provided with a suitable optical system, not shown, that is located in 
front of the photosensor for both filtering away undesired Ught and for focusing a picture on the 
light sensitive surface of the photosensor, and a Ught source 3 placed coaxially with the optical 
axis of the photosensor, i.e. located around, at all sides of, the Ught sensitive surface of the photo- 

35 sensor. The Ught sensitive suifece can, as shown, have a rectangular shape. The coaxial Ught 
source 1 includes, see Figs. 3a and 3b, a pluraUty of Ught emitting elements 3', 3" arranged m two 
groups, an first inner group and a second outer group. In the inner group the Ught emitting ele- 
ments 3' are placed close to the edges or sides of the Ught sensitive surface. In the embodiment 
shown including a rectangular Ught sensitive area three elements are placed at each long side of 
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the rectangular shape and two elements at each short side thereof. In the outer group the Ught 
emitting elements 3" are placed at a larger distance of the Ught sensitive area, outside the ele- 
ments m the inner group as seen from the center of the Ught sensitive are. In the embodiment 
shown there are as many elements in the outer group as m the inner group and each element in 
5 the outer group is placed between two elements in the inner group, as seen from the center of the 
rectangular Ught sensitive area. 

In a practical embodunent of the instaUation according to Fig. 1 the three identical light 
sources 2 each comprised seven NIR-diodes, HSDL 4230, includmg six diodes arranged synmiet- 
ricaUy around a center diode. The same types of diodes are used as the Ught emittmg elements 3', 
10 3" in the iUiraiination unit 3 designed according to Figs. 3a and 3b. 

In addition to these components only a coordinating calculation and control unit is pro- 
vided, indicated at 6, for performing the required control and evaluation associated therewith and 
calculations. Thus, the calculation and control unit controls switching the Ught sources on and 
off. Furthermore it performs processing of unages captured by the photosensor 4, in particular 
15 determination of boundary Unes, i.e. lines between fields in the pictures that have different grey 
scale intensities or colors, hi the image of an eye among other things the boundary Une is deter- 
mmed that deUmits the pupU and therefrom the center of the pupU is obtained. The latter can be 
made by for example adapting a circle or eUipse to the boundary Une that has been decided to de- 
limit the pupU. The calculation and control unit 6 can also control, as wiU be described hereinaf- 
20 ter, the data output from the photosensor 4, i.e. determine the picture elements m a captured im- 
age for which mformation is to be forwarded from the photosensor to the calculation and control ' 
unit 

In Figs. 3a and 3b it is seen how the elements in the Ught source 3 are placed around the 
photosensor 4 to be capable of iUuminatmg the eye in two different ways, by switching the Ught 

25 emittmg elements of the inner or outer group on and off. One Ught settmg position which is seen 
m Fig. 3 and in which only the elements m the mner group are switched on causes an easy ob- 
servable bright eye effect of the pupUs that are detected by the photosensor whereas the other set- 
ting position which is shown in Fig. 3b and m which only the elements in the outer group are 
switched on causes reflections from corneas without causmg any easy observable bright eye ef- 

30 feet. 

When the installation of Fig. 1 is operating, it alternates between two light settmg positions. 
Setting position (i) is provided for determining the direction in which the user is looking. In Ught 
setting position (u) the distance from the eyes of the user to the photosensor is determined, hi 
both Ught setting positions also information about the place at which the eye or eyes of the user is 
35 located as seen in the two-dunensional coordmate system of the photosensor is obtained. 

When the instaUation is in Ught settmg position (i) the mner elements 3' of the Ught source 
3 that is arranged coaxiaUy with the field of vision of the photosensor 4 are switched on m the 
settmg position that is shown in Fig. 3a and gives a strong bright eye effect for aU pupUs within 
the field of vision of the photosensor. In this Ught settmg position aU the three Ught sources 2 
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placed above the monitor 1 are switched off. i.e. only the elements in the inner group are 
switched on. The image of a pupU within the visual field of the photosensor.2 will then look as 
appears from Fig. 4a in which the pupil 5, compare Fig. 4b, is iUuminated due to the bright eye 
effect and the reflection of the Ught source 3 from the cornea is seen as a point-shaped reflection 
5 3'a, The captured image is analyzed by the calculation unit 6 and boundary lines according to Fig. 
4b are determined. In the picture c^tured for the bright eye effect there is a particularly large 
contrast between the image fields that correspond to the pupil and the iris, compare the picture of 
Fig. 5a which is c£q>tured using a different illumination, and therefore the boundary line between 
these twQ fields can be calculated with a good accuracy. This accurately determined boundary 
10 line delimits the pxxpH and therefrom the center of the pupil can then be determined, consequently 
also with a good accuracy. In the determination of the center, as has been mentioned above, adap- 
tation of an ellipse to the boundary line can be used, which is particularly important in the case 
where the pupU is only partly visible in the picture. Furthermore, in the picture the position of a 
reflection 3'a and particularly the center of the reflection are determined. Finally the vector from 
15 the center of the reflection 3'a to the center of the pupil 5 is calculated. 

To determine the gaze direction, i.e. the direction in which the user is looking, the vector 
determined according to the description above is used. The direction of this vector mdicates the 
direction in which the user is looking as taken from the optical axis of the camera and the mag- 
nitude of the vector indicates the angle in which the user looks, also taken in relation to the opti- 
20 cal axis of the camera. Furthermore, the center of the reflection 3'a provides information about 
the position of the ^e of the user as seen in two dimensions. 

When the system is in light setting position (ii), the elements 3" in the outer gro\q) in the 
Ught source 3 that is coaxial with the visual field of the photosensor are switched in the light set- 
ting position according to Fig. 3b that does not result in any clearly observable bright eye effect. 
25 At the same time the three light sources 2 placed at the upper edge of the monitor are switched on 
in this light settmg position. Fig. 5a illustrates how an eye can be imaged on to the photosensor 4 
in this setting position with a relatively low contrast between the image fields that correspond to 
different portions of the eye. Also this picture is analyzed to obtain boundary Unes, as is iUus- 
trated in Fig. 5b, and the accuracy of the determined boundary lines of fields corresponding to 
30 different portions of the eye can be lower than that of the line delimitmg the pupil according to 
Fig. 4b. However, in the picture according to Fig. 5b the different reflections appear very clearly. 
At 3'b, see the analyzed picture in Fig. 5b, thus the reflection of the Ught source 3, that is 
switched on according to the case appearing from Fig. 3b, &om the cornea is iUustrated and at 2' 
the reflections of the Ught sources 2 placed at the upper edge of the monitor from the cornea are 
35 shown. The positions of and the distances between the different reflections are determined in the 
picture and this determination can then be made with a good accuracy. The pattern of the four re- 
flections. i.e. primarily the positions of the different reflections in relation to each other, is evalu- 
ated to determine the distance of the eye from the photosensor 4. In particular the size of the pat- 
tern that is formed by the four reflections at 2', 3'b is dependent on the distance of the eye from 
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the photosensor. The longer from the photosensor the eye is located, the analler are the dimen- 
sions of the pattern, hi that way a measure of the distance at which the monitor 1 is- located from 
the eye of the user is obtamed. Also in this case information about the place where the eye of the 
user is located in two dimensions is obtained from the reflection of the coaxial light source 3 at 

Thus, when the system of Fig. 1 is in operation, it alternates between light setting position 

(i) and light setting position (ii). It requires information from these two light setting positions to 
be capable of indicating the position on a monitor at which a user is looking. In Ught setting posi- 
tion (i) the gaze direction of the user is detected and in light setting position (ii) the position of 

10 the user is detected. For a computer user, eye movements are usually significantiy much more 
rapid than movements of the head. This means that it is more important to have recentiy updated 
information about the gaze angle of the user than about the position of the user in the depth di- 
rection from the photosensor 4 to obtain a good accuracy m the determination of the point on the 
computer monitor on which the user is looking. Experiments indicate that in the case where four 

15 pictures captured in light setting position (i) axe analyzed for each picture in Ught setting position 

(ii) a system having a good accuracy is obtained. For determining the gaze point information 
from the last picture taken m light setting position (i) and from tiie last picture captured in light 
setting position (ii) is always used. 

In Fig. 2 is schanatically illustrated how the distance between images of reflections of two 
20 light sources depends on the distance of the reflecting surfece from tiie light sources and tiie de- 
tector which are assumed to be located in tiie same plane. The distance between the reflecting 
surfece and the plane through the light sources is roughly inversely proportional to the square 
root of the distance between the images. Such a relationship or a somewhat more exact relation- 
ship, as appears from Fig. 2, can be used together with absolute values obtained in an individual 

25 calibration of the installation for each user to give a sufficiently accurate distance determination 
for for example control of a cursor on the monitor when the head of the user is moving. 

The photosensor used in the installation described above is of high resolution type. In that 
way tiie user can be allowed to have a greater mobility of her/his head witiiout producing a de- 
graded accuracy. To reduce the load on the calculation unit and mcrease tiie sampling velocity of 

30 the system data from the photosensor are not more than then necessarily used. The calculation 
unit uses information about the positions where eyes have been found in previous pictures to se- 
lect the portion of the Ught sensitive surface of the photosensor tiiat is to be exposed before each 
new picture to be taken or to be used m each new picture and controls the photosensor in accor- 
dance tiierewith. If the usor's eyes are found in the preceding picture only a small portion of the 

35 picture elements of the Ught sensitive surface of the photosensor is used. Only in flie case where 
no eyes have been found data from aU picture elements of the Ught sensitive surface of the photo- 
sensor are used or in any case data representing the whole picture surfece, obtained for example 
by a down sampUng operation of data for the total pictiire area. In tiiat way a system is obtamed 
that is relatively slow m detecting eyes and the distance of eyes in the depth direction but that can 
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be made to track eyes and in particular the gaze direction with a high velocity after the eyes have 
been found. 

An eye has unique physical characteristics which togetiier with the position of tiie eye and 
the gaze direction control how a picture captured as illustrated in Fig. 4a or 5a will appear. To be 
5 capable of determining tiie position at vMch a person is looking from tiie data that can be ob- 
tamed fiom tiiese pictures tiie system can be individually caUbrated, if required, for tiie person 
tiiat is sitting in front of the monitor. In addition tiie system has to know which eye is which one, 
i.e. if tiie boundary lines calculated according to Figs. 4b and 5b are associated witii tiie left or 
right eye of the user. 

10 When tiie two eyes of a user are visible to tiie photosensor 4 it is easy to tell which eye tiiat 
is tiie left eye and which one is tiie right eye but in tiie case where only one eye has been found 
another metiiod is required to determine which eye it is. Such a metiiod will now be described 
and tiie different steps tiierein are also illustrated by tiie flow diagram of Fig. 6. The first factor 
having influence is tiie motion velocity of tiie eyes of tiie user. There is an upper limit for tiie ve- 
15 locity witii which an eye can move. It results m tiiat if tiiere is a known eye in a given position in 
a previous picture tiiis eye cannot have moved more tiian a definite distance after tiie previous 
picture was captured. Thus, if tiie distance is too large between tiie previously known eye and tiie 
eye now found, tiie eye now found is an eye different from that which was previously detected. 
Even if tiie previously known eye was tiie only known eye at that time there must have been an 
20 area witiiin which tiie otiier eye should have been located. The distance between tiie eyes of a 
user can in this way be used to judge wheflier tiie eye tiiat was not known can have moved to tiie 
position where tiie new eye was found If tiie distance is sufBcientiy small for only one of tiie 
eyes of the user the found eye must be the same one as this eye. 

If said criteria do not give information about vAAch eye has been found anotiier factor exists 
25 which can be considered. This fector is how close to tiie edge of tiie photosensor tiie eye is lo- 
cated. If tiie eye is located so close to tiie outer edge of tiie photosensor tiiat tiie otiier eye should 
be located outside tiie visual field of tiie sensor, if it was located on tiie outer side of tiie found 
eye, it is Hug tiiat either tiiis must be true or tiiat tiie otiier eye is located witiiin tiie gaze angle of 
tiie photosensor but is concealed - probably due to tiie fact tiiat tiie user has closed it. In tiiis case 
30 it can be allowed tiiat tiie system will guess tiiat tiie found eye is tiie eye tiiat implies tiiat tiie 
otiier eye is outside tiie field of vision of tiie photosensor. After tiie system has guessed which eye 
has been found a lower confidence level is set for tiie data tiiat are forwarded &om tiie system to 
show that these data are not secure. 

In Fig. 7 a flow diagram of the different steps executed in determining and tracking the eyes 
35 and gaze direction of a person is shown. In a first step 71 a pictiire is captured witii illummation 
in tight setting position (i), i.e. -wrfhien only tiie elements 3' are switched on. In tiie picture pupils 
are determined, i.e. tiieir boundary lines, and reflections of tiie tight source. In tiie next block 73 
the procedure according to Fig. 6 is executed to determine which eye is which one. In step 75 it is 
tiien decided whetiier two eyes have been found in tiie picture. If it is true, a block 77 is executed 
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in which a picture is captured with the iUumination in setting position (ii), i.e. with the elements 
3" switched on and the diodes 2 activated. In this picture only information about the areas around 
the found eyes is forwarded to the unit 6. Reflections and thereby images of eyes in the picture 
are determined. Thereupon, in a block 79 again the procedure according to Fig. 6 is executed to 

5 determine yMch eye is which one. In the next step 81 it is decided whether two eyes have been 
found in the picture. If it is true case, in a block 83 a calculation of the point observed by the eyes 
is performed in the unit 6. Then a block 85 is executed in which again a picture is captured with 
the illumination in Ught setting position (i). i.e. with only the elements 3' switched on. For the 
captured image only information for areas around found eyes is transmitted to the unit 6. From 

10 the restricted pictorial information pupUs and reflections of the light source are determined in the 
same way as in step 71. Then in a block 87 again the procedure according to Fig. 6 is executed to 
determine which eye is which one. In the next step 89 it is decided whether two eyes have been 
found in the picture. If it is true, in a block 91 a calculation of the point observed or gazed at by 
the eyes is executed in the same way as in block 83. Thereupon in a block 93 it is decided 

15 whether the four last pictures captured were captured with the iUumination in Ught setting posi- 
tion (ii). If it is not true, the block 85 is again executed, and otherwise block 77. If it is deter- 
mined in any of the blocks 75, 81 and 89 that two eyes have not been found, the block 71 is again 
executed. 

While specific embodiments of the invention have been illustrated and described herein, it 
20 is reaUzed that numerous additional advantages, modifications and changes wiU readily occur to 
those skilled in the art Therefore, the invention in its broader aspects is not limited to the specific 
details, representative devices and illustrated examples shown and described herein. Accordingly, 
various modifications may be made without departing from the spirit or scope of the general in- 
ventive concept as defined by the appended claims and their equivalents. It is therefore to be un- 
25 derstood that the upended claims are intended to cover all such modifications and changes as fall 
within a true spirit and scope of the inventioiL 



